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Abstract 

We study the ergodicity of backward product of stochastic and doubly stochas- 
tic matrices by introducing the concept of absolute infinite flow property. We show 
that this property is necessary for ergodicity of any chain of stochastic matrices, by 
defining and exploring the properties of a rotational transformation for a stochastic 
chain. Then, we establish that the absolute infinite flow property is equivalent to 
ergodicity for doubly stochastic chains. Furthermore, we develop a rate of conver- 
gence result for ergodic doubly stochastic chains. We also investigate the limiting 
behavior of a doubly stochastic chain and show that the product of doubly stochas- 
tic matrices is convergent up to a permutation sequence. Finally, we apply the 
results to provide a necessary and sufficient condition for the absolute asymptotic 
stability of a discrete linear inclusion driven by doubly stochastic matrices. 

1 Introduction 

The study of forward product of an inhomogeneous chain of stochastic matrices is closely 
related to the limiting behavior, especially ergodicity, of inhomogeneous Markov chains. 
The earliest study on the forward product of inhomogeneous chains of stochastic matrices 
is the work of Hajnal in [Tj. Motivated by a homogeneous Markov chain, Hajnal formu- 
lated the concepts of ergodicity in weak and strong senses for inhomogeneous Markov 
chains and developed some sufficient conditions for both weak and strong ergodicity of 
such chains. Using the properties of scrambling matrices that were introduced in pQ, 
Wolfowitz [2] gave a condition under which all the chains driven from a finite set of 
stochastic matrices are strongly ergodic. In his elegant work [3], Shen gave geometric 
interpretations and provided some generalizations of the results in [T] by considering vec- 
tor norms other than || • ||ooj which was originally used in [T] to measure the scrambleness 
of a matrix. 

The study of backward product of row-stochastic matrices, however, was motivated 
by different applications all of which were in search of a form of a consensus between 
a set of processors, individuals, or agents. DeGroot [I] studied such a product (for 
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a homogeneous chain) as a tool for reaching consensus on a distribution of a certain 
unknown parameter among a set of agents. Later, Chatterjee and Seneta [5] provided 
a theoretical framework for reaching consensus by studying the backward product of an 
inhomogeneous chain of stochastic matrices. Motivated by the theory of inhomogeneous 
Markov chains, they defined the concepts of weak and strong ergodicity in this context 
and showed that those two properties are equivalent. Furthermore, they developed 
the theory of coefficients for ergodicity. Motivated by some distributed computational 
problems, in [S], Tsitsiklis and Bertsekas studied such a product from the dynamical 
system point of view. In fact, they considered a dynamics that enables an exogenous 
input as well as delays in the system. Through the study of such dynamics, they gave 
a more practical conditions for a chain to ensure the consensus. The work in [6] had a 
great impact on the subsequent studies of distributed estimation and control problems. 

The common ground in the study of both forward and backward products of stochas- 
tic matrices are the chains of doubly stochastic matrices. By transposing the matrices 
in such a chain, forward product of matrices can be transformed into backward product 
of the transposes of the matrices. Therefore, in the case of doubly stochastic matrices, 
any property of backward products translates to the same property for forward prod- 
ucts. However, since the transposition of a row-stochastic matrix is not necessarily a 
row-stochastic matrix, the behavior of the forward and backward products of stochastic 
matrices can be quite different in general. 

Here, we study the backward product of a chain of stochastic matrices in general, 
and then focus on doubly stochastic matrices in particular. We start by introducing 
a concept of absolute infinite flow property as a refinement of infinite flow property, 
proposed in our earlier work in [7]. We demonstrate that the absolute infinite flow 
property is more stringent than the infinite flow property, by showing that a sequence of 
permutation matrices cannot have the absolute infinite flow property, while it may have 
the infinite flow property. We introduce a concept of rotational transformation, which 
plays a central role in our exploration of the relations between the absolute infinite 
flow property and ergodicity of a stochastic chain. In fact, using the properties of the 
rotational transformation, we show that the absolute infinite flow property is necessary 
for ergodicity of any stochastic chain. However, even though this property requires a 
rich structure for a stochastic chain, it is still not sufficient for the ergodicity, which we 
illustrate on an example. In pursue of chains for which this property is sufficient for 
ergodicity, we come to identify a class of decomposable chains, which have a particularly 
simple test for the absolute infinite flow property. Upon exploring decomposable chains, 
we show that doubly stochastic chains are decomposable by using Birkhoff-von Neumann 
theorem. Finally, we show that the ergodicity and absolute infinite flow property are 
equivalent for doubly stochastic chains and provide a convergence rate result for ergodic 
doubly stochastic chains by establishing a result on limiting behavior of doubly stochastic 
chain that need not be ergodic. We conclude by considering a discrete linear inclusion 
system driven by a subset of stochastic or doubly stochastic matrices. We provide a 
necessary and sufficient conditions for the absolute asymptotic stability of such discrete 
linear inclusions. 

This work is a continuation of our earlier work in [7J [8] where we have studied ran- 
dom backward products for independent, not necessarily identically distributed, random 
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matrix processes with row-stochastic matrices. In [£], we have introduced infinite flow 
property and shown that this property is necessary for ergodicity of any chain of row- 
stochastic matrices. Moreover, we have shown that this property is also sufficient for 
ergodicity of such chains under some additional conditions on the matrices. In [8J, we 
have extended some of our results from [7j to a larger class of row-stochastic matrices 
by using some approximations of chains. The setting in [TJ [8] has been within random 
matrix processes with row-stochastic matrices. Unlike our work in [7J [8] , the work in 
this present paper is focused on deterministic sequences of row-stochastic matrices. Fur- 
thermore, the new concept of the absolute infinite flow property is demonstrated to be 
significantly stronger than the infinite flow property. Finally, our line of analysis in this 
paper relies on the development of some new concepts, such as rotational transformation 
and decomposable chains. Our result on convergence rate for ergodic doubly stochastic 
chains extends the rate result in [H] to a larger class of doubly stochastic chains, such 
as those that may not have uniformly bounded positive entries or uniformly bounded 
diagonal entries. 

The main new results of this work include: (1) Formulation and development of 
absolute infinite flow property. We show that this property is necessary for ergodicity 
of any stochastic chain, despite the fact that it is much stronger than infinite flow 
property. (2) Introduction and exploration of rotational transformation of a stochastic 
chain with respect to a permutation chain. We establish that rotational transformation 
preserves several properties of a chain, such as ergodicity and absolute infinite flow 
property. (3) Establishment of equivalence between ergodicity and absolute infinite 
flow property for doubly stochastic chains. We accomplish this through the use of the 
Birkhoff-von Neumann decomposition of doubly stochastic matrices and properties of 
rotational transformation of a stochastic chain. (4) Development of a rate of convergence 
for ergodic doubly stochastic chains. (5) Establishment of the limiting behavior for 
doubly stochastic chains. We prove that a product of doubly stochastic matrices is 
convergent up to a permutation sequence. (6) Development of necessary and sufficient 
conditions for the absolute asymptotic stability of a discrete linear inclusion system 
driven by doubly stochastic matrices. 



The structure of this paper is as follows: in Section 1.1 , we introduce the notation 
that we use throughout the paper. In Section [2j we discuss the concept of ergodicity and 
introduce the absolute infinite flow property. In Section [3j we introduce and investigate 
the concept of rotational transformation. Using the properties of this transformation, 
we show that this property is necessary for ergodicity of any stochastic chain. Then, in 
Section [4] we introduce decomposable chains and study their properties. In Section [5j 
we study the product of doubly stochastic matrices by exploring ergodicity, rate of 
convergence, and the limiting behavior in the absence of ergodicity. In Section [6j we 
study the absolute asymptotic stability of a discrete inclusion system driven by doubly 
stochastic matrices. We summarize the development in Section [7} 

1.1 Notation and Basic Terminology 

We view all vectors as columns. For a vector x, we write Xi to denote its zth entry, 
and we write x > (x > 0) to denote that all its entries are nonnegative (positive). 
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We use x T to denote the transpose of a vector x. We write ||x|| to denote the standard 
Euclidean vector norm i.e., ||x|| = x 1- We use e « ^° denote the vector with the 

ith entry equal to 1 and all other entries equal to 0, and we write e for the vector with 
all entries equal to I. We say that a vector v G lR m is stochastic if Vi > for all % and 
Y^i=\ Vi = l- For a given set C and a subset S of C, we write 5cCto denote that S is 
a proper subset of C. A set S C C such that 5" 7^ is referred to as a nontrivial subset 
of C. We write [to] to denote the integer set {1, . . . , to}. For a set S C [m], we let \S\ 
be the cardinality of the set S and 5 be the complement of S with respect to [to], i.e., 
S = {ie[m]\i<£ S}. 

We denote the identity matrix by /. For a matrix A, we use Aij to denote its (2, j)th 
entry, A4 and A J to denote its ith row and jth column vectors, respectively, and A T 
to denote its transpose. We write \\A\\ for the matrix norm induced by the Euclidean 
vector norm. A matrix A is row-stochastic when all of its rows are stochastic vectors. 
Since we deal exclusively with row-stochastic matrices, we refer to such matrices simply 
as stochastic. A matrix A is doubly stochastic when both A and A T are stochastic. We 
say that a chain of matrices {A(k)} is a stochastic chain if A(k) is a stochastic matrix 
for all k > 0. Similarly, we say that {A(k)} is a doubly stochastic chain if A(k) is a 
doubly stochastic matrix for all k > 0. We may refer to a stochastic chain or a doubly 
stochastic chain simply as a chain when there is no ambiguity on the nature of the 
underlying chain. 

For an to x to matrix A, we use $^i<j A^ to denote the summation of the entries A^ 
over all i,j G [to] with i < j. Given a nonempty index set S C [to] and a matrix A, we 
write As to denote the following summation: 

A s= J2 A v + J2 Ai r 

i£S,j&S i&S,j&S 

Note that A s satisfies A s = J2 ieS jgs(Aj + A??)- 

An to x to matrix P is a permutation matrix if it contains exactly one entry equal 
to 1 in each row and each column. Given a permutation matrix P, we use P(S) to 
denote the image of an index set S C [to] under the permutation P; specifically P(S) = 
{i G [to] I Pij = 1 for some j G S}. We note that a set S C [to] and its image P(S) 
under a permutation P have the same cardinality, i.e., \S\ = \P(S)\. Furthermore, for 
any permutation matrix P and any nonempty index set S C [to], the following relation 
holds: 

ieP(S) j&s 

We denote the set of toxto permutation matrices by £? m - Since there are to! permutation 
matrices of size to, we may assume that the set of permutation matrices is indexed, i.e., 
S^m = {P^ I 1 < £ < to!}. Also, we say that {P(k)} is a permutation sequence if 
P(k) G 0P m for all k > 0. The sequence {/} is the permutation sequence {P(k)} with 
P(k) = I for all k, and it is referred to as the trivial permutation sequence. 

We view a directed graph G as an ordered set (V, E) where V is a finite set of vertices 
(the vertex set) and E C V x V is the edge set of C For two vertices u, v G V, we often 
use u — > v to denote the edge («, v) e E. A cycle in G = (V, i?) is a sequence of vertices 
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Mo, Mi, ... , u r -i, u r = uq such that u T — > u T+ i for all r G {0, . . . , r — 1}. We denote such 
a cycle by u — > «i — > ■ ■ ■ u r -\ — > u r — u . Throughout this paper, we assume that 
the vertices of a cycle are distinct. Note that a loop (v,v) G E is also a cycle. We say 
that a directed graph G is cycle-free when G does not contain a cycle. We say that 
u — > U\ — V Ui — > ■ ■ ■ is a walk in G if Ui — > u i+ i for all i > 0. 

2 Ergodicity and Absolute Infinite Flow 

In this section, we discuss the concepts of ergodicity and infinite flow property, and we 
introduce a more restrictive property than infinite flow, which will be a central concept 
in our later development. 

2.1 Ergodic Chain 

Here, we define the ergodicity for a backward product of a chain {A(k)} of m x m 
stochastic matrices A(k). For k > s > 0, let A(k : s) = A(k - l)A(k - 2) • • • A(s). We 
use the following definition for ergodicity. 

Definition 1. We say that a chain {A(k)} is ergodic if 

lim A(k : s) = ev T (s) for all s > 0, 

where v(s) is a stochastic vector for all s > 0. 

In other words, a chain {A(k)} is ergodic if the infinite backward product ■ ■ ■ A(s + 
2)A(s + l)A(s) converges to a matrix with identical rows, which must be stochastic 
vectors since the chain {A(k)} is stochastic. 

To give an intuitive understanding of the backward product A(k : s) of a stochastic 
chain {A(k)}, consider the following simple model for opinion dynamics for a set [m] = 
{1, . . . , m} of m agents. Suppose that at time s, each of the agents has a belief about a 
certain issue that can be represented by a scalar G R. Now, suppose that agents' 
beliefs evolve in time by the following dynamics: at any time k > s, the agents meet 
and discuss their beliefs about the issue and, at time k + 1, they update their beliefs by 
taking convex combinations of the agents' beliefs at the prior time k, i.e., 

m 

Xi(k + 1) = y~] Aij{k)xj{k) for all % G [m], (1) 
j'=i 

with Y^jLiAijik) = 1 for all i G [m], i.e., A(k) is stochastic. In this opinion dynamics 
model, we are interested in properties of the chain {A(k)} that will ensure the m agents 
reach a consensus on their beliefs for any starting time s > and any initial belief 
profile x(s) G M m . Formally, we want to determine some restrictions on the chain which 
will guarantee that lim^oo (xi(k) —Xj{k)) = for any i, j G [m] and any s > 0. As 
proven in [5], this limiting behavior is equivalent to ergodicity of {A(k)} in the sense of 
Definition [TJ 
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* 2 (3) = 1 4 = 1 4 x 2 (2) = |fi 4 

Figure 1: The trellis graph of the 2x2 chain {A(k)} with weights A{k) given in Eq. ([3]). 

One can visualize the dynamics in Eq. Q using the trellis graph associated with 
a given stochastic chain. The trellis graph of a stochastic chain {A(k)} is an infinite 
directed weighted graph G = (V, E, {A(k)}), with the vertex set V equal to the infinite 
grid [m] x Z + and the edge set 

E = {((j,k),(i,k + l))\j,ie[m], k>0}. (2) 

In other words, we consider a copy of the set [m] for any time k > and we stack these 
copies over time, thus generating the infinite vertex set V = {(i, k) \ i G [m], k > 0}. 
We then place a link from each j G [m] at time k to every i G [m] at time k + 1, i.e., 
a link from each vertex (j, k) G V to each vertex (i, + 1) G V. Finally, we assign 
the weight Ay(fc) to the link ((j,k),(i,k + 1)). Now, consider the whole graph as an 
information tunnel through which the information flows: we inject a scalar 2j(0) at each 
vertex (i, 0) of the graph. Then, from this point on, at each time k > 0, the information 
is transfered from time to time k + 1 through each edge of the graph that acts as a 
communication link. Each link attenuates the in- vertex's value with its weight, while 
each vertex sums the information received through the incoming links. One can observe 
that the resulting information evolution is the same as the dynamics given in Eq. Q. 
As an example, consider the 2x2 static chain {A(k)} with A(k) defined by: 



A(k) 



1 3 

4 4 

3 I 

4 4 



for k>0. (3) 



The trellis graph of this chain and the resulting dynamics is depicted in Figure [TJ 
2.2 Infinite Flow and Absolute Infinite Flow 

Here, we discuss the infinite flow property, as introduced in [7j, and a more restrictive 
version of this property which turned out to be necessary for ergodicity of stochastic 



chains, as shown later on in Section 3.1 



We start by recalling the definition of the infinite flow property from [7]. 
Definition 2. A stochastic chain {A(k)} has the infinite flow property if 

oo 

'^^As(k) = oo for all nonempty S C [m], 

A:=0 

where A s (k) = J2 i& s,jes ( A ij( k ) + A jii k ))- 
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Graphically, the infinite flow property requires that in the trellis graph of a given 
model {A(k)}, the weights on the edges between S x Z + and S x Z + sum up to infinity. 

To illustrate the infinite flow property, consider the opinion dynamics in Eq. Q. 
One can interpret A^{k) as the credit that agent i gives to agent j's opinion at time 
k. Therefore, the sum X^es j E § Aij(k) can be interpreted as the credit that the agents' 
group S C [m] gives to the opinions of the agents that are outside of S (the agents in 
S) at time k. Similarly, J2ies j&s Aij(k) is the credit that the agents' group S gives to 
the agents in group S. The intuition behind the concept of infinite flow property is that 
without having infinite accumulated credit between agents' groups S and S, we cannot 
have an agreement among the agents for any starting time s of the opinion dynamic 
and for any opinion profile x(s) of the agents. In other words, the infinite flow property 
ensures that there is enough information flow between the agents as time passes by. 

We have established in [7] that the infinite flow property is necessary for ergodicity 
of a stochastic chain, as restated below for later use. 

Theorem 1. |7|/ The infinite flow property is a necessary condition for ergodicity of any 
stochastic chain. 

Although the infinite flow property is necessary for ergodicity, this property alone 
is not strong enough to disqualify some stochastic chains from being ergodic, such as 
chains of permutation matrices. As a concrete example consider a static chain {A(k)} 
of permutation matrices A(k) given by 

for k > 0. (4) 

As a remedy for this situation, in [7j we have imposed some additional conditions, in- 
cluding some feedback-type conditions, on the matrices A(k) that eliminate permutation 
matrices. Here, we take a different approach. 

Specifically, we will require a stronger infinite flow property by letting the set S in 
Definition [2] vary with time. In order to do so, we will consider sequences {S(k)} of 
index sets S(k) C [m] with a form of regularity in the sense that the sets S(k) have 
the same cardinality for all k. In what follows, we will reserve notation {S(k)} for the 
sequences of index sets S(k) C [m\. Furthermore, for easier exposition, we define the 
notion of regularity for {S(k)} as follows. 

Definition 3. A sequence {S(k)} is regular if the sets S(k) have the same (nonzero) 
cardinality, i.e., \S{k) \ = \S(0)\ for all k > and \S(0)\ ^ 0. 

The nonzero cardinality requirement in Definition [3] is imposed only to exclude the 
trivial sequence {S(k)} consisting of empty sets. 

Graphically, a regular sequence {S(k)} corresponds to the subset {(i,k) \ i G 
S(k), k > 0} of vertices in the trellis graph associated with a given chain. As an 
illustration, let us revisit the 2x2 chain given in Eq. (|4]). Consider the regular {S(k)} 
defined by 

q(u\ _ / {^} if k i s even ) 
[ ' ~ \ {2} if k is odd. 



A(k) 



1 

1 
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Figure 2: The trellis graph of the permutation chain in Eq. Q. For the regular 
sequence {S(k)}, with S(k) = {1} if k is even and S(k) = {2} otherwise, the vertex set 
{(i,k) | i G S(k), k > 0} is marked by black vertices. The flow F({A(k)}; {S(k)}), as 
defined in d6l), corresponds to the summation of the weights on the dashed edges. 



The vertex set {(i, k) \ i G S(k), k > 0} associated with {S(k)} is shown in Figure [2] 

Now, let us consider a chain {A(k)} of stochastic matrices A{k). Let {S{k)} be any- 
regular sequence. At any time k, we define the flow associated with the entries of the 
matrix A(k) across the index sets S(k + 1) and S(k) as follows: 

A S ( k +i),s(k)(k) = J2 A v( k ) + J2 forfc>0. (5) 

i£S(k+l),j£S(k) iES(k+l),jeS{k) 

The flow As(k+i),s(k)(k) could be viewed as an instantaneous flow at time k induced by 
the corresponding elements in the matrix chain and the index set sequence. Accordingly, 
we define the total flow of a chain {A(k)} over {S(k)}, as follows: 

oo 

F({A(k)};{S(k)}) = J2Mk+i),S(k)(k). (6) 

k=0 

We are now ready to extend the definition of the infinite flow property to time varying 
index sets S(k). 

Definition 4. A stochastic chain {A(k)} has the absolute infinite flow property if 
F({A(k)}; {S(k)}) = oo for every regular sequence {S(k)}. 

Note that the absolute infinite flow property of Definition [4] is more restrictive than 
the infinite flow property of Definition [2] In particular, we can see this by letting the set 
sequence {S(k)} be static, i.e., S(k) = S for all k and some nonempty S C [m]. In this 
case, the flow As(fc+i),S(fc)(^) across the index sets S(k + 1) and S(k) as defined in Eq. ^ 
reduces to A s (k), while the flow F({A(k)}; {S(k)}) in Eq. @ reduces to J2T=o A s(k)- 
This brings us to the quantities that define the infinite flow property (Definition [2]). 
Thus, the infinite flow property requires that the flow across a trivial regular sequence 
{S} is infinite for all nonempty S C [m], which is evidently less restrictive requirement 
than that of Definition |4j In light of this, we see that if a stochastic chain {A(k)} has 
the absolute infinite property, then it has the infinite flow property. 

The distinction between the absolute infinite flow property and the infinite flow 
property is actually much deeper. Recall our example of the chain {A(k)} in Eq. Q, 
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which demonstrated that a permutation chain may posses the infinite flow property. 
Now, for this chain, consider the regular sequence {S(k)} with S(2k) = {1} and S{2k + 
1) = {2} for k > 0. The trellis graph associated with {A(k)} is shown in Figure [2j 
where {S(k)} is depicted by black vertices. The flow F({A(k)}; {S(k)}) corresponds to 
the summation of the weights on the dashed edges in Figure [2j which is equal to zero in 
this case. Thus, the chain {A(k)} does not have the absolute flow property. 

In fact, while some chains of permutation matrices may have the infinite flow prop- 
erty, it turns out that no chain of permutation matrices has the absolute infinite flow 
property. In other words, the absolute infinite flow property is strong enough to filter 
out the chains of permutation matrices in our search for necessary and sufficient condi- 
tions for ergodicity which is a significant distinction between the absolute infinite flow 
property and the infinite flow property. To formally establish this, we turn our attention 
to an intimate connection between a regular index sequence and a permutation sequence 
that can be associated with the index sequence. 

Specifically, an important feature of a regular sequence {S(k)} is that it can be 
obtained as the image of the initial set 5(0) under a certain permutation sequence 
{P(k)}. This can be seen by noting that we can always find a one-to-one matching 
between the indices in S(k) and S(k + 1) for all k, since \S(k)\ = \S(k + 1)|. As a 
result, any set S(k) is an image of a finitely many permutations of the initial set 5(0); 
formally S(k) = P(k — 1) • • • P(1)P(0)(5(0)). Therefore, we will refer to the set S(k) as 
the image of the set 5(0) under {P(k)} at time k. Also, we will refer to the sequence 
{S(k)} as the trajectory of the set 5(0) under {P(k)}. 

As a first consequence of the definition, we show that no chain of permutation ma- 
trices has the absolute infinite property. 

Lemma 1. For any permutation chain {P(k)}, there exists a regular sequence {S(k)} 
for which 

F({P(k)};{S(k)}) = 0. 

The result follows by letting {S(k)} be the trajectory of a nonempty set 5(0) C [m] 
under the permutation {P(k)}. 

3 Necessity of Absolute Infinite Flow for Ergodicity 

As discussed in Theorem [TJ the infinite flow property is necessary for ergodicity of a 
stochastic chain. In this section, we show that the absolute infinite flow property is 
actually necessary for ergodicity of a stochastic chain despite the fact that this property 
is much more restrictive than the infinite flow property. We do this by considering a 
stochastic chain {A(k)} and a related chain, say {B(k)}, such that the flow of {A(k)} 
over a trajectory translates to a flow of {B(k)} over an appropriately defined trajectory. 
The technique that we use for defining the chain {B(k)} related to a given chain {A(k)} 
is developed in the following section. Then, in Section |3.2[ we prove the necessity of the 
absolute infinite flow for ergodicity. 
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3.1 Rotational Transformation 



Rotational transformation is a process that takes a chain and produces another chain 
through the use of a permutation sequence {P(k)}. Specifically, we have the following 
definition of the rotational transformation with respect to a permutation chain. 

Definition 5. Given a permutation chain {P(k)}, the rotational transformation of an 
arbitrary chain {A(k)} with respect to {P(k)} is the chain {B(k)} given by 

B(k) = P T (k + 1 : 0)A(k)P(k : 0) for k > 0, 

where P(0 : 0) = /. We say that {B(k)} is the rotational transformation of {A(k)} by 
{P(k)}. 

The rotational transformation has some interesting properties for stochastic chains 
which we discuss in the following lemma. These properties play a key role in the subse- 
quent development, while they may also be of interest in their own right. 

Lemma 2. Let {A(k)} be an arbitrary stochastic chain and {P(k)} be an arbitrary 
permutation chain. Let {B(k)} be the rotational transformation of {A(k)} by {P(k)}. 
Then, the following statements are valid: 

(a) The chain {B(k)} is stochastic. Furthermore, 

B(k : s) = P T (k : 0)A(k : s)P(s : 0) for any k > s > 0, 
where P(0 : 0) = L 

(b) The chain {A(k)} is ergodic if and only if the chain {B(k)} is ergodic. 

(c) For any regular sequence {S(k)} for {A(k)}, there exists another regular sequence 
{T(k)} for {B(k)} such that B T{k+1)T(k )(k) = A S ( k +i)s(k)(k) . Also, for any regular 
sequence {T(k)} for {B{k)}, there exists a regular sequence {S{k)} for {A{k)} with 
A s{k+ i)s(k)(k) = B T{k+l)T{k) {k). Inparticular, F({A(k)}; {S(k)}) = F({B(k)}; {T(k)}) 
and hence, {A(k)} has the absolute infinite flow property if and only if {B(k)} has 
the absolute infinite flow property. 

(d) For any S C [m] and k > 0, we have As(k+i),s(k){k) = Bs{k), where S{k) is the 
image of S under {P(k)} at time k, i.e., S(k) = P(k : 0)(S'). 

Proof, (a) By the definition of B(k), we have B(k) = P T (k + l : 0)A(k)P(k : 0). Thus, 
B(k) is stochastic as the product of finitely many stochastic matrices is a stochastic 
matrix. 

The proof of relation B(k : s) = P T (k : 0)A(k : s)P(s : 0) proceeds by induction on 
k for k > s and an arbitrary but fixed s > 0. For k — s + 1, by the definition of B(s) 
(see Definition g, we have B(s) = P T (s + 1 : 0)A(s)P(s : 0), while B(s + 1 : s) = B(s) 
and A(s + 1 : s) = A(s). Hence, B(s + 1, s) = P T (s + 1 : 0)A(s + 1 : s)P(s : 0) which 
shows that B(k,s) = P T (k : 0)A(k : s)P(s : 0) for k = s + 1, thus implying that the 
claim is true for k — s + 1. 



10 



Now, suppose that the claim is true for some k > s, i.e., B(k,s) = P T {k : 0)A(k : 
s)P(s : 0) for some k > s. Then, for k + 1 we have 

B(h + l:s) = B(k)B(k : s) = B(k) (P T (k : 0)A(k : s)P(s : 0)) , (7) 

where the last equality follows by the induction hypothesis. By the definition of B(k), 
we have B(k) = P T (k + 1 : 0)A(k)P(k : 0), and by replacing B(k) by P T (k + 1 : 
0)A(k)P(k : 0) in Eq. Q, we obtain 

B{k + l:s)= (P T {k + 1 : 0)A(k)P(k : 0)) (P T (k : 0)A{k : s)P{s : 0)) 
= P T (k + 1 : 0)A(k) (P(k : 0)P T (k : 0)) A(k : s)P(s : 0) 
= P T (k + 1 : 0)A(k)A(k : s)P(a : 0), 

where the last equality follow from P T P = I which is valid for any permutation matrix 
P, and the fact that the product of two permutation matrices is a permutation matrix. 
Since A(k)A(k : s) = A(k + 1 : s), it follows that 

B(k + l:s)= P T (k + 1 : 0)A(k + 1 : s)P(s : 0), 

thus showing that the claim is true for k + 1. 

(b) Let the chain {A(k)} be ergodic and fix an arbitrary starting time to > 0. Then, for 
any e > 0, there exists a sufficiently large time N t > t , such that the rows of A(k : t ) 
are within e-vicinity of each other; specifically ||v4j(fc : t ) — Aj(k : £ )|| — e for any 
k > N e and all i,j G [m]. We now look at the matrix B{k : t ) and its rows. By part 
(a), we have for all k > to, 

B(k : t ) = P T (k : 0)A(k : t )P(t : 0). 

Furthermore, the ith row of B(k : to) can be represented as e[B(k : to)- Therefore, the 
norm of the difference between the zth and jth row of B{k : to) is given by 

\\Bi(k : U)-B 3 (k : t )|| = ||(e t -e J ) T 5(A; : t )|| = \\(e l -e,) T P T (k : 0)A(k : t )P(t : 0)||. 

Letting eaj^ = P(k : 0)ej for any i G [m], we further have 

\\B t (k : to) - Si(A: : t )|| = ||(e i(fc) - e m ) T A{k : t )P(t : 0)|| 

= \\(A i(k) (k : t ) - A j(k) (k : t ))P(t : 0)|| 

= \\A m (k : t ) - A m (k : t )||, (8) 

where the last inequality holds since P(t : 0) is a permutation matrix and \\Px\\ = \\x\\ 
for any permutation P and any x G M. m . Choosing k > N t and using : t ) — Aj(k : 

^o)|| ^ e for an y k > N e and all i,j G [m], we obtain 

\\Bi(k : to) — Bj(k : to)\\ < e for any k > N e and all i,j G [m]. 

Therefore, it follows that the ergodicity of {A(k)} implies the ergodicity of {B(k)}. 
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For the reverse implication we note that A(k : to) — P{k '■ 0)B(k : to)P T (to '■ 0), 
which follows by part (a) and the fact PP T = PP T = I for any permutation P. The rest 
of the proof follows a line of analysis similar to the preceding case, where we exchange 
the roles of B{k : t ) and A{k : t ). 

(c) Let {S(k)} be a regular sequence. Let T{k) be the image of S{k) under the permu- 
tation P T (k : 0), i.e. T(k) = P T (k : 0)(S(k)). Note that |T(jfe)| = \S(k)\ for all k > and 
since {S(k)} is a regular sequence, it follows that {T(k)} is also a regular sequence. Now, 
by the definition of rotational transformation we have A(k) = P(k + 1 : 0)B(k)P T (k : 0), 
and hence: 

ejA(k)e, = £ e ^ k + 1 : 0)B(k)P T (k : 0)}e 3 
ies(k+i),j£§(k) ieS{k+i),jeS(k) 

J2 ejB(k)e r 

iET(k+l),jeT(k) 

Similarly, we have Ei e 5(fc+i),j 6 s(fc) e I A ( k ) e j = E ie f (k+i),jeT(k) e I B ( k )ej. Next, note 
that 

A S (k+i)S(k)(k) = Yl e J A (k)ej+ e J A (k)ej, 

ieS{k+l),je§(k) i£S(k+l),jeS{k) 

B T (k+i)T{k)(k) = Y efB(k)ej+ ^ e I B (k)ej. 
ier(fc+i)jef(fc) ief(fc+i),jeT(fc) 

Hence, we have A S (k+i)S(k){k) = B T ( k+1)T ( k) (k), implying 

F({A(k)};{S(k)}) = F({B(k)};{T(k)}). 

For the converse, for any regular sequence {T(fc)}, if we let S(k) = P{k : 0)(T(k)) and 
use the same line of argument as in the preceding case, we can conclude that {S(k)} is 
a regular sequence and A S ( k +i)s(k)(k) = B T{k+1)Tik) (k). Therefore, F({A(k)}; {S(k)}) = 
F({B(k)}; {T(k)}) implying that {A(k)} has the absolute infinite flow property if and 
only if {B(k)} has the absolute infinite flow property. 

(d) If {P(k)} is such that S(k) is the image of 5* under {P(k)} at any time k > 0, by 
part (c), it follows that A S (k+i)S(k)(k) = B T ^ k+1)T(k) (k), where T(k) = P T (k : 0)(S(k)). 
Since S(k) is the image of S under {P(k)} at time k, it follows that P T (k : 0)(S(k)) = S 
in view of P T (k : 0)P(k : 0) = L Thus, T(k + 1) = S and by part (c) we obtain 
A S (k+i)s(k)(k) = B s (k). Q.E.D. 

As listed in Lemma [2| the rotational transformation has some interesting properties: 
it preserves ergodicity and it preserves the absolute infinite flow property, by interchang- 
ing the flows. We will use these properties intensively in the development in this section 
and the rest of the paper. 

3.2 Necessity of Absolute Infinite Flow Property 

We now establish the necessity of the absolute infinite flow property for ergodicity of 
stochastic chains. The proof of this result relies on Lemma [2] 
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Theorem 2. The absolute infinite flow property is necessary for ergodicity of any 
stochastic chain. 

Proof. Let {A(k)} be an ergodic stochastic chain. Let {S(k)} be any regular sequence. 
Then, there is a permutation sequence {P(k)} such that {S(k)} is the trajectory of 
the set S(0) C [m] under {P{k)}, i.e., 3(k) = P{k : 0)(S(0)) for all k, where P(0 : 
0) = I. Let {-£?(&;)} be the rotational transformation of {A(k)} by the permutation 
sequence {P(k)}. Then, by Lemma [^Q, the chain {B(k)} is stochastic. Moreover, by 
Lemma [2||b]) , the chain {B(k)} is ergodic. Now, by the necessity of the infinite flow 
property (Theorem [T]), the chain {B(k)} should satisfy 

oo 

Bs(k) = oo for all nonempty S C [m]. (9) 

k=0 

Therefore, by Lemma [2j|d]) , Eq. Q with S = 5(0) implies 

oo 

7^ As(fc+i),s(fc)(fc) = oo, 

fc=0 

thus showing that {A(k)} has the absolute infinite flow property. Q.E.D. 

The converse statement of Theorem [2] is not true generally, namely the absolute infi- 
nite flow need not be sufficient for the ergodicity of a chain. We reinforce this statement 
later in Section [4] (Corollary [T]). Thus, even though the absolute infinite flow property 
imposes a lot of structure for a chain {A(k)}, by requiring that the flow of {A(k)} over 
any regular sequence {S(k)} be infinite, this is still not enough to guarantee ergodicity 
of the chain. However, as we will soon see, it turns out that this property is sufficient 
for ergodicity of the doubly stochastic chains. 

4 Decomposable Stochastic Chains 

In this section, we consider a class of stochastic chains, termed decomposable, for which 
verifying the absolute infinite flow property can be reduced to showing that the flows 
over some specific regular sequences are infinite. We explore some properties of this class 
which will be also used in later sections. 

Here is the definition of a decomposable chain. 

Definition 6. A chain {A(k)} is decomposable if {A(k)} can be represented as a non- 
trivial convex combination of a permutation chain {P(k)} and a stochastic chain {A(k)}, 
i.e., there exists a 7 > such that 

A(k) = ^P{k) + (1 - i)A{k) for all k > 0. (10) 

We refer to {P(k)} as a permutation component of {A(k)} and to 7 as a mixing coeffi- 
cient for {A(k)}. 
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An example of a decomposable chain is a chain {A(k)} with uniformly bounded 
diagonal entries, i.e., with Au(k) > 7 for all k > and some 7 > 0. Instances of 
such chains have been studied in [TUl [TTJ [TJl UHl HH EH US]- In this case, A(k) = 
7/ + (1 - 7)A(ife) where A(k) = ^ r (A(k) - 7/). Note that A(k) - 7/ > 0, where the 
inequality is to be understood component- wise, and (A(k) — jl)e = (1 — 7)e, which 
follows from A(k) being stochastic. Therefore, A(k) is a stochastic matrix for any k > 
and the trivial permutation {/} is a permutation component of {A(k)}. Later, we will 
show that any doubly stochastic chain is decomposable. 

An example of a non- decomposable chain is the static chain {A(k)} defined by 



A(k) = A(0) 



1 
1 



for all k > 0. 



This chain is not decomposable since otherwise, we should have A(0) = 7-P(0) + (1 — 
7)^4(0) for some permutation matrix P(0). Since the second column of A(0) is the zero 
vector, the second column of P(0) should also be zero which contradicts the fact that 
P(0) is a permutation matrix. 

We have some side remarks about decomposable chains. The first remark is an 
observation that a permutation component of a decomposable chain {A(k)} need not to 
be unique. An extreme example is the chain {A(k)} with A{k) = ^ee T for all k > 0. 
Since ^ee T = ^Xl^i 3 ®, any sequence of permutation matrices is a permutation 
component of {A(k)}. Another remark is about a mixing coefficient 7 of a chain {A(k)}. 
Note that if 7 > is a mixing coefficient for a chain {A(k)}, then any £ e (0,7] is also 



a mixing coefficient for {A(k)} } as it can be seen from the decomposition in Eq. (10). 

An interesting property of any decomposable chain is that if such a chain is rota- 
tionally transformed with respect to its permutation component, the resulting chain has 
trivial permutation component {/}. This property is established in the following lemma. 

Lemma 3. Let {A(k)} be a decomposable chain with a permutation component {P(k)} 
and a mixing coefficient 7 . Let {B(k)} be the rotational transformation of {A(k)} with 
respect to {P(k)}. Then, the chain {B(k)} is decomposable with a trivial permutation 
component {1} and a mixing coefficient 7. 

Proof. Note that by the definition of a decomposable chain (Definition [6]), we have 

A(k) = 7 P(A;) + (1 - i)A{k) for all k > 0, 

where P(k) is a permutation matrix and A(k) is a stochastic matrix. Therefore, 

A(k)P(k : 0) = -fP(k)P(k : 0) + (1 - -f)A(k)P{k : 0). 

By noticing that P(k)P(k : 0) = P(k + 1:0) and by using left-multiplication with 
P T (k + 1 : 0), we obtain 

P T (k + l : 0)A(k)P(k : 0) = 7 P T (A: + 1 : 0)P(k+l : 0) + (l~7)P T (A; + l : 0)A(k)P(k : 0). 

By the definition of the rotational transformation (Definition [5]), we have B(k) = P T (k + 
1 : 0)A(k)P(k : 0). Using this and the fact P T P = I for any permutation matrix P, we 
further have 

B(k) = 7/ + (1 - 7)P T (A; + 1 : 0)A(k)P(k : 0). 
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Define B(k) = P T (k + l : 0)A(k)P(k : 0) and note that each B(k) is a stochastic matrix. 
Hence, 

B{k)= 1 I + (l- 1 )B{k), 

thus showing that the chain {B (k)} is decomposable with the trivial permutation com- 
ponent and a mixing coefficient 7. Q.E.D. 

In the next lemma, we prove that the absolute infinite flow property and the infinite 
flow property are one and the same for decomposable chains with a trivial permutation 
component. 

Lemma 4. For a decomposable chain with a trivial permutation component, the infinite 
flow property and the absolute infinite flow property are equivalent. 

Proof. By definition, the absolute infinite flow property implies the infinite flow property 
for any stochastic chain. For the reverse implication, let {A(k)} be decomposable with 
a permutation component {/}. Also, assume that {A(k)} has the infinite flow property. 
To show that {A(k)} has the absolute infinite flow property, let {S(k)} be any regular 
sequence. If S(k) is constant after some time to, i.e., S(k) = S(t ) for k > t and some 
to > 0, then 

00 00 

A s(k+i),s(k)(k) = Y A s(t )(k) = 00, 

k=to k=to 

where the last equality holds since {A(k)} has infinite flow property and ^!°=o A s(t )(k) 
is finite. Therefore, if S(k) = S (to) for k > t Q , then we must have YlT=o A s(k+i),s(k)(k) = 
00. 

If there is no t > with S(k) = S(t ) for k > to, then we must have S(k r +1) 7^ S(k r ) 
for an increasing time sequence {k r }. Now, for an i 6 S(k r ) \ S(k r + 1) 7^ 0, we have 
^4s(fc r +i),5(fc r )(^r) > Au(k r ) since i G S(k r + 1). Furthermore, Au(k) > 7 for all k since 
{A(k)} has the trivial permutation sequence {/} as a permutation component with a 
mixing coefficient 7. Therefore, 

CO OO OO 

> YA s{kr+1)iS ( kr )(k r ) > 7^1 = 00. 

fc=0 r=0 r=0 

All in all, F({A(k)}, {S(k)}) = 00 for any regular sequence {S(k)} and, hence, the chain 
{A(k)} has the absolute infinite flow property. Q.E.D. 

Lemma [4] shows that the absolute infinite flow property may be easier to verify for the 
chains with a trivial permutation component, by just checking the infinite flow property. 
This result, together with Lemma [3] and the properties of rotational transformation 
established in Lemma [2j provide a basis to show that a similar reduction of the absolute 
infinite flow property is possible for any decomposable chain. 
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Theorem 3. Let {A(k)} be a decomposable chain with a permutation component {P(k)}. 
Then, the chain {A(k)} has the absolute infinite flow property if and only if F({A(k)}; {S(k)}) 
oo for any trajectory {S(k)} under {P(k)}, i.e., for all S(0) C [m] and its trajectory 
{S{k)} under {P{k)}. 

Proof. Since, by the definition, the absolute infinite flow property implies F({A(k)}; {S(k)}) = 
oo for any regular sequence {S(k)}, it suffice to show that F({A(k)}; {S(k)}) = oo 
for any trajectory {S(k)} under {P(k)}. To show this, let {B(k)} be the rotational 
transformation of {A(k)} with respect to {P(k)}. Since {A(k)} is decomposable, by 
Lemma[3j it follows that {B(k)} has the trivial permutation component {/}. Therefore, 
by Lemma [4] {B(k)} has the absolute infinite flow property if and only if it has the 
infinite flow property, i.e., 

oo 

Bs(k) = oo for all nonempty S C [m). (11) 

fc=0 



By Lemma 2 1 we have Bs(k) = As(k+i),S(k)(k), where S(k) is the image of S(0) = S 



under the permutation {P(k)} at time k. Therefore, Eq. ( JTT] ) holds if and only if 

oo 

^2 As(k+i)s(k){k) = oo, 



fc=0 



which in view of F({A(k)}; {S(k)}) = JXo Mk+i)S(k){k) shows that F({A(k)}; {S(k)}) 
oo. Q.E.D. 

Another direct consequence of Lemma [4] is that the absolute infinite flow property is 
not generally sufficient for ergodicity. 



Corollary 1. The absolute infinite flow property is not a sufficient condition for ergod- 
icity. 



Proof. Consider the following static chain: 



A(k) 



1 

ill 

3 3 3 
1 



for k > 0. 



It can be seen that {A(k)} has the infinite flow property. Furthermore, it can be seen that 
{A(k)} is decomposable and has the trivial permutation sequence {/} as a permutation 
component. Thus, by Lemma|4j the chain {A(k)} has the absolute infinite flow property. 
However, {A(k)} is not ergodic. This can be seen by noticing that the vector v = 
(1,t;,0) t is a fixed point of the dynamics x{k + 1) = A{k)x{k) with x(0) = v, i.e., 
v = A(k)v for any k > 0. Hence, {A(k)} is not ergodic. Q.E.D. 

Although the absolute infinite flow property is a stronger necessary condition for 
ergodicity than the infinite flow property, Corollary [l] demonstrates that the absolute 
infinite flow property is not yet strong enough to be equivalent to ergodicity. 
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5 Doubly Stochastic Chains 



In this section, we focus on the class of the doubly stochastic chains. We first show that 
this class is a subclass of the decomposable chains. Using this result and the results 
developed in the preceding sections, we establish that the absolute infinite flow property 
is equivalent to ergodicity for doubly stochastic chains. 

We start our development by proving that a doubly stochastic chain is decomposable. 
The key ingredient in this development is the Birkhoff-von Neumann theorem ( [T7] page 
527) stating that a doubly stochastic matrix is a convex combination of permutation 
matrices. More precisely, by the Birkhoff-von Neumann theorem, a matrix A is doubly 
stochastic if and only if A is a convex combination of permutation matrices, i.e., 

m! 

where Yl™=i = 1 an< ^ — f° r £ £ [ml]. Here, we assume that all the permutation 
matrices are indexed in some manner (see notation in Section [T]). 

Now, consider a sequence {A{k)} of doubly stochastic matrices. By applying the 
Birkhoff-von Neumann theorem to each A(k), we have 

to! 

#) = ^#)?(9 (12) 

where X^iQfcW = 1 an< ^ — f° r all £ G [m!] and fc > 0. Since ^£=i<??(^) = 1 

and ^(fc) > 0, there exists a scalar 7 > ^7 such that for every k > 0, we can find 
£(k) G [m!] satisfying q^(k)(k) > 7. Therefore, for any time k > 0, there is a permutation 
matrix P(k) = P^ k ^ such that 

to! 

A(k) = 7 P(jfe) + a e (Jfe)P® = 7P(ife) + (1 - 7)i(Jfe), (13) 
€=1 

where 7 > is a time-independent scalar and = j^— J2T=i Oi^(k)P^\ 



The decomposition of A(k) in Eq. (13) fits the description in the definition of de- 



composable chains. Therefore, we have established the following result. 

Lemma 5. Any doubly stochastic chain is decomposable. 

In the light of Lemma[5j all the results developed in Section|4]are applicable to doubly 
stochastic chains. Another result that we use is the special instance of Theorem 6 in [7] 
as applied to doubly stochastic chains. Any doubly stochastic chain that has the trivial 



permutation component {/} (i.e., Eq. (13) holds with P(k) = I) fits the framework of 



Theorem G in p]. We restate this theorem for convenience. 

Theorem 4. Let {A(k)} be a doubly stochastic chain with a permutation component 
{I}. Then, the chain {A{k)} is ergodic if and only if it has the infinite flow property. 
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Now, we are ready to deliver our main result of this section, showing that the er- 
godicity and the absolute infinite flow are equivalent for doubly stochastic chains. We 
accomplish this by combining Theorem [3] and Theorem |4j 

Theorem 5. A doubly stochastic chain {A(k)} is ergodic if and only if it has the absolute 
infinite flow property. 

Proof. Let {P(k)} be a permutation component for {A(k)} and let {B(k)} be the 
rotational transformation of {A(k)} with respect to its permutation component. By 
Lemma [3j {B(k)} has the trivial permutation component {/}. Moreover, since B{k) = 
P T (k + l : 0)A(k)P(k : 0), where P T (k + 1 : 0), A(k) and P(k : 0) are doubly stochastic 
matrices, it follows that {B(k)} is a doubly stochastic chain. Therefore, by Theorem |4j 
it follows that {B(k)} is ergodic if and only if it has the infinite flow property. Then, by 
Lemma [2jjd]) , the chain {B(k)} has the infinite flow property if and only if {v4(A;)} has 
the absolute infinite flow property. Q.E.D. 

Theorem[5]provides an alternative characterization of ergodicity for doubly stochastic 
chains, under only requirement to have the absolute infinite flow property. We note that 
Theorem [5] does not impose any other specific conditions on matrices A{k) such as 
uniformly bounded diagonal entries or uniformly bounded positive entries, which have 
been typically assumed in the existing literature (see for example [TQl [Hi [12], [131 El E3 

EE]). 

We observe that the absolute infinite flow typically requires verifying that the infinite 
flow exists along every regular sequence of index sets. However, to use Theorem |5j we do 
not have to check the infinite flow for every regular sequence. This reduction in checking 
the absolute infinite flow property is due to Theorem [3j which shows that in order to 
assert the absolute infinite flow property for doubly stochastic chains, it suffices that 
the flow over some specific regular sets is infinite. We summarize this observation in the 
following corollary. 

Corollary 2. Let {A{k)} be a doubly stochastic chain with a permutation component 
{P(k)}. Then, the chain is ergodic if and only if F({A(k)}; {S(k)}) = 00 for all trajec- 
tories {S(k)} of subsets S(0) C [m] under {P(k)}. 

5.1 Rate of Convergence 

Here, we explore a rate of convergence result for an ergodic doubly stochastic chain 
{A(k)}. In the development, we adopt a dynamical system point of view for the chain 
and we consider a Lyapunov function associated with the dynamics. The final ingredient 
in the development is the establishment of another important property of rotational 
transformation related to the invariance of the Lyapunov function. 

Let {A(k)} be a doubly stochastic chain and consider the dynamic system driven 
by this chain. Specifically, define the following dynamics, starting with any initial point 
x(0) G M m , 

x(k + 1) = A(k)x(k) for k > 0. (14) 
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With this dynamics, we associate a Lyapunov function V(-) defined as follows: 

m 

V(x) = ^(xi - xf for x G R m , (15) 



where x = —e x. This function has often been used in studying ergodicity of doubly 
stochastic chains (see for example [15], [J5J [TBI E])- 

We now consider the behavior of the Lyapunov function under rotational transfor- 
mation of the chain {A(k)}, as given in Definition [5j It emerged that the Lyapunov 
function V is invariant under the rotational transformation, as shown in forthcoming 
Lemma [BJ We emphasize that the invariance of the Lyapunov function V holds for arbi- 
trary stochastic chain; the doubly stochastic assumption is not needed for the underlying 
chain. 

Lemma 6. Let {A(k)} be a stochastic chain and {P(k)} be an arbitrary permutation 
chain. Let {B(k)} be the rotational transformation of {A(k)} by {P(k)}. Let {x(k)} 
and {y{k)} be the dynamics obtained by {A{k)} and {B{k)}, respectively, with the same 
initial point y(0) = x(0) where x(0) G lR m is arbitrary. Then, for the function V(-) 



defined in Eq. (15) we have 

V(x(k)) = V(y(k)) for all k > 0. 

Proof. Since {y(k)} is the dynamic obtained by {B(k)}, there holds for any k > 0, 

y(k) = B(k - l)y(k - 1) = . . . = B(k - 1) • • • B(l)B(0)y(0) = B(k : 0)j/(0). 

By Lemma §0), we have B(k : 0) = P T (k : 0)A(k : 0)P(0 : 0) with P(0 : 0) = J, 
implying 

y (k) = P T (k : 0)A(k : 0)y(0) = P T (k : 0)A(k : 0)x(0) = P T (k : O)z(Jfe), (16) 

where the second equality follows from y(0) = x(0) and the last equality follows from 
the fact that {x(k)} is the dynamic obtained by {A(k)}. Now, notice that the function 



V(-) of Eq. (15) is invariant under any permutation, that is V(Px) = V(x) for any 



permutation matrix P. In view of Eq. (16), the vector y(k) is just a permutation of 



x(k). Hence, V(y(k)) = V(x(k)) for all k > 0. Q.E.D. 

Consider an ergodic doubly stochastic chain {A(k)} with a trivial permutation com- 
ponent {/}. Let t = and for any 5 G (0, 1) recursively define t q , as follows: 



t-i 

t +i = arg min min > A s (k) > 5, (17) 

t>t a +l Sc\m] 



t=t 



where the second minimum in the above expression is taken over all nonempty subsets 
S C [m\. Basically, t q is the first time t > when the accumulated flow from 
t = tq-i + 1 to t — t q exceeds S over every nonempty S C [m]. We refer to the sequence 
{t q } as accumulation times for the chain {A(k)}. We observe that, when the chain 
{A(k)} has infinite flow property, then t q exists for all q > 0, and any 5 > 0. 

Now, for the sequence of time instances {t q }, we have the following rate of conver- 
gence result. 
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Lemma 7. Let {A(k)} be an ergodic doubly stochastic chain with a trivial permutation 
component {/} and a mixing coefficient 7 > 0. Also, let {x(k)} be the dynamics driven 
by {A(k)} starting at an arbitrary point x(0). Then, for all q > 1, we have 



where t q is defined in ( |17[ ). 

The proof of Lemma [7] is based on the proof of Theorem 10 in [9] and Theorem 6 
in [7], and it is provided in Appendix. 

Using the invariance of the Lyapunov function under rotational transformation and 
the properties of this transformation, we can establish a result analogous to Lemma [7j 
for an arbitrary ergodic chain of doubly stochastic matrices {A(k)}. In other words, we 
can extend Lemma [7] to the case when the chain {A(k)} does not necessarily have the 
trivial permutation component {/}. To do so, we appropriately adjust the definition of 
the accumulation times {t q } for this case. In particular, we let 5 > be arbitrary but 
fixed, and let {P(k)} be a permutation component of an ergodic chain {A(k)}. Next, 
we let to = and for q > 1, we define t q as follows: 

t-i 

t q+l = arg min min VA s(w)s(fe) (l:) > 6, (18) 

t>t q +l S(0)C m ^ 
t=t« 

where {S(k)} is the trajectory of the set S(0) under {P(k)}. 
We have the following convergence result. 

Theorem 6. Let {A(k)} be an ergodic doubly stochastic chain with a permutation com- 
ponent {P(k)} and a mixing coefficient 7 > 0. Also, let {x(k)} be the dynamics driven 
by {A{k)} starting at an arbitrary point x(0). Then, for all q > 1, we have 



where t q is defined in (18) 



Proof. Let {B(k)} be the rotational transformation of the chain {A(k)} with respect to 
{P(k)}. Also, let {y{k)} be the dynamics driven by chain {B(k)} with the initial point 
y(0) = x(0). By Lemma |3j {B(k)} has the trivial permutation component {/}. Thus, 
by Lemma [7], we have for all q > 1, 

V(y(t q )) <(l- % ) V(y(t q ^)). 

Now, by Lemma |2|[d]) , we have A S ( k+ i) S (k)(k) = B s (k). Therefore, the accumulation 
times for the chain {A(k)} are the same as the accumulation times for the chain {B(k)}. 
Furthermore, according Lemma [6j we have V(y(k)) = V(x(k)) for all k > and, hence, 
for all q > 1, 

V(x(t q )) <(l- ^f (1 " % ) 

Q.E.D. 
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5.2 Doubly Stochastic Chains without Absolute Flow Property 



So far we have been concerned with doubly stochastic chains with the absolute infinite 
flow property. In this section, we turn our attention to the case when the absolute flow 
property is absent. In particular, we are interested in characterizing the limiting behavior 
of backward product of a doubly stochastic chain that does not have the absolute infinite 
flow. 

Since a doubly stochastic chain is decomposable, Theorem [3] is applicable, so by 
this theorem when the chain {A(k)} does not have the absolute infinite flow property, 
there holds F({A(k)}; {S(k)}) < oo for some 5(0) C [m] and its trajectory under 
a permutation component {P(k)} of {A(k)}. This permutation component will be 
important so we denote it by V. Furthermore, for this permutation component, let 
Gp = ([m],£p) be the undirected graph with the edge set £pP given by 



where {i(k)} and {j(k)} are the trajectories of the sets 5(0) = {i} and 5(0) = {j}, 
respectively, under the permutation component {P(k)}; formally, e^k) = P(k : 0)ej and 
e m = P(k : 0)ej for all k with P(0 : 0) = I. We refer to the graph = (M,££?) as 
the infinite flow graph of the chain {A(k)}. When the permutation component {P(k)} 
is trivial, we use G°° to denote G™. 

By Theorem [3j we have that G^ is connected if and only if the chain has the ab- 
solute infinite flow property. Theorem 6 in [8] shows that for a chain with the trivial 
permutation component {/}, the connectivity of G°° is closely related to the limiting 
matrices of the product A(k : to), as k — > oo (in the case of the trivial permutation 
component {/}, we have i(k) = i for all i G [m] and k > 0). The following result is just 
an implication of Theorem 6 in [S] for the special case of doubly stochastic chains that 
have the trivial permutation component. 

Theorem 7. Let {A(k)} be a doubly stochastic chain with the trivial permutation com- 
ponent {I}. Then, for any starting time to, the limit A°°(to) = linifc-^oo A(k : to) exists. 
Moreover, the ith row and the jth row of A°°(to) are identical for any to > 0, i.e., 
lim^oo \\Ai(k : t ) — Ajik : £ )|| = 0, if and only if i and j belong to the same connected 
component of G°° . 

Using Lemma |2j Lemma [3] and Theorem [7], we prove a similar result for the case of 
a general doubly stochastic chain. 

Theorem 8. Let {A(k)} be a doubly stochastic chain with a permutation component 
{P(k)}. Then, for any starting time to > 0, the product A(k : to) converges up to 
a permutation of its rows; i.e., there exists a permutation sequence {Q{k)} such that 
lim^oo Q{k)A{k : to) exists for any t Q > 0. Moreover, for the trajectories {i{k)} and 
{j{k)} of 5(0) = {i} and 5(0) = {j}, respectively, under the permutation component 
{P(k)}, we have 



£. 




lim ||Aj(fc)(/c : to) — Aj^)(k : £q)|| = for any starting time to, 
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if and only if i and j belong to the same connected component ofGp. 

Proof. Let {B(k)} be the rotational transformation of {A(k)} by the permutation com- 
ponent {P(k)}. As proven in Lemma |3j the chain {B(k)} has the trivial permutation 
component. Hence, by Theorem [7j the limit B°°(t ) = Yim^^ B{k : t ) exists for any 
to > 0. On the other hand, by Lemma [2]ja]) , we have 

B(k : t ) = P T (k : 0)A(k : t )P(t : 0) for all k > to- 
Multiplying by P T (to '■ 0) from the right, and using PP T = I which is valid for any 
permutation matrix P, we obtain 

B(k : t )P(t : 0) T = P T (k : 0)A(k : t ) for all k > t . 

Therefore, lim^oo B{k : t )P(t : 0) T always exists for any starting time to since B(k : 
t )P T (t : 0) is obtained by a fixed permutation of the columns of B{k : t ). Thus, if we 
let Q(k) = P T {k : 0), then lim^oo Q{k)A{k : t ) exists for any t which proves the first 
part of the theorem. 

For the second part, by Theorem [7| we have lim^oo \\Bi{k : t ) — Bj(k : t )|| = 
for any to > if and only if i and j belong to the same connected component of the 
infinite flow graph of {B(k)}. By the definition of the rotational transformation, we 
have B(k : t ) = P T (k : 0)A(k : t )P(t : 0). Then, for the ith and jth row of B(k : t ), 
we have according to Eq. (Tsl) : 



\\Bi{k : t ) - Bj(k : t )|| = \\A i{k) {k : t ) - A m {k : t )||, 

where e^k) = P{k : 0)ej and e^) = P(k : 0)e 3 - for all k. Therefore, lim^oo \\Bi{k : 
t ) - Bj(k : t ) || = if and only if lim^oo \\A i(k )(k : t ) - A j{k) (k : t )|| = 0. Thus, 
lim^oo ||v4j( fc )(fc : t ) — Aj(u\{k : t )|| = for any t > if and only if i and j belong 
to the same connected component of the infinite flow graph of {B(k)}. The last step is 
to show that the infinite flow graph of {B(k)} and {A(k)} are the same. This however, 
follows from 

oo oo oo 

J2 B v(k) = J2 e ^ pT ( k + 1 : 0)Mk)P(k : 0)] ej = ^ A i{k+1)m {k). 

k=0 k=0 k=0 

Q.E.D. 



By Theorem[8j for any doubly stochastic chain {A(k)} and any fixed to, the sequence 
consisting of the rows of A(k : to) converges to an ordered set of m points in the 
probability simplex of lR m , as k approaches to infinity. In general, this is not true for an 
arbitrary stochastic chain. For example, consider the stochastic chain 



A(2k) 



1 
1 
1 



A(2k + 1) 



1 
1 
1 



for all k > 0. 



For this chain, we have A(2k : 0) = A(2k) and A(2k + 1:0) = A(2k + 1). Hence, 
depending on the parity of k, the set consisting of the rows of A{k : 0) alters between 
{(1,0,0), (1,0,0), (0,0,1)} and {(1, 0, 0), (0, 0, 1), (0, 0, 1)} and, hence, never converges 
to a unique ordered set of 3 points in IR 3 . 
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6 Application to Switching Systems 



Motivated by the work in [Ty\, here we study an application of the developed results to 
the stability analysis of switching systems that are driven by a subset of stochastic or 
doubly stochastic matrices. 

Consider an arbitrary collection ^ of m x m stochastic matrices. Let us say that 
a chain {A(k)} is in ^# if A(k) G ^ for all k > 0. Our main goal is to characterize 
conditions on the collection M such that 

lim A(k : 0) = — ee T , (19) 
fc->oo m 

for any chain {A(k)} in j$ . If {A(k)} is a doubly stochastic chain and we let A(k) = 
A{k) — —ee T , then we have A(k : 0) = A(k : 0) — — ee T . Therefore, relation (19) holds 



for any doubly stochastic chain {A(k)} in ^ if and only if the (translated) discrete 
linear inclusion system j$ = {A G ^ \ A — —ee T } is absolutely asymptotically stable 
as defined in 



Also, note that if the relation (19) holds for any chain {A(k)} in then we should 
have linifc-j.oo A{k : to) — — ee T for all to > and {A(k)} in ^ . This simply follows from 
the fact that if {A(k)} is in then any truncated chain {A(k)}k>t of {A(k)} should 



also be in jfa ' . Thus, relation (19) holds for all chains {A(k)} in ^ if and only if j% 
only consists of ergodic chains. 

Based on the preceding discussion and the definition of the absolute asymptotic 
stability for discrete linear inclusion systems [19] , let us define the absolute asymptotic 
stability for a collection of stochastic matrices. 

Definition 7. We say that a collection ^ of stochastic matrices is absolutely asymp- 
totically stable if every chain {A(k)} in ^ is ergodic. 

To derive necessary and sufficient conditions for the absolute asymptotic stability, 
we define the zero-flow graph associated with the collection ^ . This graph is defined 
over a vertex set consisting of all subsets of the set [m], which is denoted by &([m}). 

Definition 8. Given a collection ~# , let G (^) = (^([m]), £ (M)) be a directed graph 
with the vertex set £P([m]) and the edge set £ (^0 defined by: 

£oM0 = {(S,T) \S,Tc [m], \S\ = \T\ ± 0, inf A TS = 0}. 

In other words, an edge (S,T) is in £ (^) if the subsets S, T C [m] are non-trivial 
and have the same cardinality, and more importantly, there exists a sequence of matrices 
A^ G ~# such that Agj, — > as k goes to infinity. 

For the forthcoming discussion, let us say that the collection has the absolute 
infinite flow property if every chain {A(k)} in ^ has the absolute infinite flow property. 
The following result provides a link between the absolute infinite flow property and the 
non-existence of cycle in the graph G (^)- 

Lemma 8. A collection ^ has the absolute infinite flow property if and only if its 
zero-flow graph G (^) is cycle- free. 



23 



Proof. Suppose that Gq(M) has a cycle S — Sq —>■•••—>■ S r — S where r > 1 and 
(S T , S T+ i) G Sq(^) for all r G {0, . . . , r — 1}. By the definition of the zero-flow graph, 
for each r G {0, . . . , r — 1}, there exists a sequence of matrices {A^ T \k)} in ^ such that 

Now, consider the chain 

{...,^ 1 )(1),^°)(1),^'- 1 )(0),...,A( 1 )(0),A( )(0)}, 
or more precisely, the chain {A{k)} defined by 

A(k) = A^\[ k -\), 
r 

where [a\ is the largest integer that is less than or equal to a. Next, consider the regular 
sequence {T(k)} defined by 

T(k) = S k _ [ k l for all k > 0. 

Note that, by the definition of the zero-flow graph, if So — > S\ — > ■ ■ ■ — > S r is a cycle in 
Gq{^) it follows that |5o| = • • • = |SV-i| and hence, {T(k)} is a regular sequence. For 
the chain {A(k)} and the regular sequence {T(k)}, we have: 

oo 

F({A(k)};{T(k)}) = J2M k +i)T(k)(k) 

fc=0 

- Z^ /l k+i-L*J,*-L*J U r- U 

fc=0 
x> r— 1 | 

= EE^ = 2r<o °- 

fc=0 r=0 

Thus, does not have the absolute infinite flow property. 

Now, assume that M does not have the absolute infinite flow property. Then, we 
have F({A(k)}] {T(k)}) < oo for some chain {A(k)} in M and a regular sequence 
{T(k)}. Consider the complete directed graph Kno), associated with the set T(0) and 
given by i^T(o) — (Vtoo), Vrto) x ^t(o)) where vertex set Vtiq) is the set of all subsets of 
[m] with cardinality |T(0)|. Now for the regular set {T(k)}, consider the infinite walk 
T(0) — > T(l) — > T(2) —>■••• in K T ( y Since i^T(o) is a finite graph, it has finitely many 
cycles and one of its cycles must be traced infinitely many times by this infinite length 
walk. Let So — >■ S\ —)■••• — >■ SV-i — > S r = Sq be such a cycle. Then, there exists an 
increasing sequence of time instances < to < t\ < • ■ • such that T(tk + r) = S T for all 
t G {0, . . . , r} and all k > 0. Therefore, 

oo r— 1 oo r— 1 

X] S ^Sr+iSr (*fc + T ) = A T(t k +r+l)T(t k +r) (*fc + r) 

fc=0 r=0 fc=0 r=0 

oo 

< £A T(Jfc+1)T(fc) (fc) < < oo. 

fc=0 
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As a result, we have lim^oo As T+1 s T (tk + r) = for all r e {0, . . . , r — 1}, implying that 
(SV, SV+i) ^ £o(-^0 f° r ai l T an d hence, G («^) contains a cycle. Q.E.D. 

Based on Lemma [8] and the results established in the preceding sections, the following 
result emerges immediately. 

Theorem 9. A collection */# is absolutely asymptotically stable only if its zero-flow 
graph Gq(^#) is cycle-free. 

Proof. By Theorem [2j ^ is absolutely asymptotically stable only if M has the absolute 
infinite flow property. At the same time, by Lemma |8j j$ has the absolute infinite flow 
property if and only if Go(^#) is cycle-free. Q.E.D. 

Now, let us focus on the case when the collection ^tft contains only doubly stochastic 
matrices. In this case, the reverse implication of Theorem thrm:aasnecessityresult also 
holds. 

Theorem 10. A collection j$ of doubly stochastic matrices is absolutely asymptotically 
stable if and only if its zero- flow graph Gq(^) is cycle-free. 

Proof. By Theorem |5j is absolutely asymptotically stable if and only if ^# has the 
infinite flow property, which by Lemma [8] holds if and only if Gq(^) is cycle- free. 
Q.E.D. 

Suppose that, in addition, we further restrict the matrices in the collection ^ to the 
matrices resulting in chains {A(k)} with trivial permutation component {/}. Then, we 
can have an alternative way of asserting the absolute asymptotic stability of j& . 

Theorem 11. Let jf( be a collection of doubly stochastic matrices such that any chain 
in jtft has the trivial permutation component {I}. Then, jtft is absolutely asymp- 
totically stable if and only if Gq{^) is loop-less, which in turn holds if and only if 
infAe^r,sc[m] A s > 0. 

Proof. Let {A(k)} be a chain in ^# with the trivial permutation component. Then, by 
Theorem [4j such a chain is ergodic if and only if it has the infinite flow property, i.e. 
SfcLo Ass(k) = oo for any non-trivial S C [m]. Using the same lines of argument as in 
the proof of Theorem [9| it can be shown that this happens if and only if inf^g^ Ass > 
for any non-trivial S C [m]. But this happens if and only if G (^#) is loop- less. Q.E.D. 

7 Conclusion 

In this paper, we studied backward product of a chain of stochastic and doubly stochastic 
matrices. We introduced the concept of absolute infinite flow property and studied its 
relation to ergodicity of the chains. In our study, a rotational transformation of a chain 
and the properties of the transformation played important roles in the development. We 
showed that this transformation preserves many properties of the original chain. Based 
on the properties of rotational transformation, we proved that absolute infinite flow 
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property is necessary for ergodicity of a stochastic chain. Moreover, we showed that this 
property is also sufficient for ergodicity of doubly stochastic chains. Then, we developed a 
rate of convergence for ergodic doubly stochastic chains. Moreover, we characterized the 
limiting behavior of a doubly stochastic chain in the absence of ergodicity. Finally, using 
the established results, we derived necessary and sufficient conditions for the stability of 
switching systems driven by stochastic and doubly stochastic matrices. 
Proof of Lemma [7j 

Proof. We prove the assertion for q = 1. The case of an arbitrary q > follows similarly. 
Let x(0) G M m be an arbitrary starting point and let {x(k)} be the dynamics driven 
by the doubly stochastic chain {A(k)}. Without loss of generality, assume that x\(0) < 
. . . < x m (0). By the definition of £1, we have ^^Zo As{k) > 5 for any S C [m]. Thus 
by Lemma 12 in [7], we have 

E E (Mk) + Mk)) Mk) - x 3 {k)f > \ m 5><+i(*) - ^c*)) 3 - 

fe=0 «<J £=1 



On the other hand, by Lemma 11 in [7], we have 

m 

^(o)) = E(^(°)- a; ( )) 2 

<m(x m (0)-X!(0)) 2 

m— 1 

<m(m -l)E(^+i(0)-^(0))' 



=i 

m— 1 



£=1 



(20) 



^■"('"- 1 ) 2 ^ (0) L 1 (o) ^<^ (0) -^ 0))3 - (21) 



Therefore, using Eq. (20) and Eq. (21), we obtain 

E E (MQ + MQ) (*«(*) - ^(^)) 2 > j^~_^ 2 V-(x(0)). (22) 

k=0 i<j y ' 

Now, by Lemma 4 in [9], we have that 

V(x(k)) = V(x(k - 1)) -J2 H ij( k - l)(xi(k - 1) - xj(k - l)) 2 , 

i<j 

where #(&) = A T (k)A(k). Thus, it follows 

V(x(ti)) = V(x(0)) -^^^W(#)-^)) 2 

fc=0 i<j 

> v(x(o)) - 7 E E + (**(*) - x ^)) 2 - ( 23 ) 

fc=0 i<j 
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In the last inequality we use the following relation: 

m 

H »( k ) = ^A H {k)A e ^{k) > A^A^k) + A^A^k) > -fiAjik) + Aji(k)), 
i=\ 

where the last inequality holds since {A(k)} has the trivial permutation component. 
Therefore, using Eq. (22) and Eq. (23), we conclude that 



V {x f_ < V(x(0)) - lj V (x 



7(5(1 ~5f 
m(m — iy 



V(x(0)). 



Q.E.D. 
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